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^ ! 1 Introduction 

Following Plotkin [P] we say that the automorphism x of the group G is a nil- automorphism 
if, for every g £ G, there exists n = n(g) such that [g, n x] = 1 (the commutator is taken in 
the holomorph of G and, following the usual notation, the element [g, n x] is defined inductively 
by [g,o,x] = g and, when n > 0, [g, n x] = [[g, n -ix],x]). If the integer n can be chosen 
independently of g, then x is said to be unipotent. Nil and unipotent automorphisms can be 
regarded as a natural extension of the concept of Engel element, since a nil-automorphism x 
^r) ' is just a left Engel element in G{x). Another way to look at nil-automorphisms, is to consider 

them as a generalization of unipotent automorphisms of vector spaces. For these reasons there 
are several natural questions that can be asked about nil- automorphisms, which are suggested 
by known facts about Engel groups or unipotent linear groups. 

-^ ' The Engel condition is, however, rather weak and it is often necessary to add some additional 

hypothesis in order to investigate this property. Similar difficulties arise when we study groups of 
nil-automorphisms. In this paper we shall focus on groups satisfying some finiteness conditions. 

The first result we prove is a consequence of a classical theorem of Baer [B]. Recall that a 
k>( ■ group G satisfies Max if every subgroup of G is finitely generated. 
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Theorem A If G satisfies Max, and H is any group of nil- automorphisms of G, then [G, H] 
is contained in the Fitting radical of G and H stabilizes a finite subnormal series in G. In 
particular H is nilpotent. 

An interesting consequence of Theorem A is that groups of of nil- automorphisms of finite 
groups are nilpotent. On the other hand every finite nilpotent group has a faithful representation 
as a group of nil-automorphisms of a suitable finite group, so that the following question seems 
natural: 

Question 1 Is it true that every finite group of nil- automorphisms is nilpotent? 

Although the question is still far from a general solution, we provide a partial answer by 
restricting ourselves to a particular class of groups. Namely the following theorem can be proved. 



2 Carlo Casolo and Orazio Puglisi 

Theorem B Let G be a locally graded group, and H a finite group of nil- automorphisms of 
G. Then H is nilpotent. 

The definition of locally graded groups will be recalled in the next section, but it is worth 
remarking that many interesting types of groups (e.g. locally nilpotent groups, locally finite 
groups, residually finite groups) are locally graded. 

A natural hypothesis, suggested by the "linear" setting, is to require that for some n, the 
identity [g,„a;] = 1 holds for all g £ G and x £ H < Aut(G). In this case we say that if is a 
group of n-unipotent automorphisms. It is then reasonable to ask to what extent the linear case 
can be generalized. 

Question 2 Let G be a group and H < Aut(G). Assume that, for some n, the identity 
[g,n x ] = 1 holds for all g £ G and x £ H. Under which hypothesis on G is it true that H is 
(locally) nilpotent? 

A well known result of Wilson |W1] says that finitely generated residually finite n-Engel 
groups are nilpotent, suggesting that Question 2 may have an affirmative answer for groups G 
in this class. It is easy to see that, rather than looking at residually finite groups, it is better to 
work with profinite groups, in order to take advantage of their well developed theory. We get 
the following theorem 

Theorem C Let G be a profinite group and H an abstract finitely generated group of n- 
unipotent automorphisms. Assume that G has a basis of open normal subgroups whose members 
are normalized by H. Then F = cIq{\G : H]) is pro-nilpotent and H is nilpotent. 

In particular theorem C applies when the profinite group G is finitely generated because, in 
that case, G has a basis whose elements are characteristic subgroups. Since any residually finite 
group can be embedded into its profinite completion, it is not hard, using theorem C, to derive 
a similar result for groups in this class. Another immediate corollary of Theorem C is the result 
of Wilson stated before. 

Finally we wish to point to the reader's attention the recent work of Crosby and Traustason 
on Engel groups. In particular the results contained in [CT], will be needed at some stage of 
our proofs. 

2 Groups with residual properties 

In order to investigate groups with residual properties, we need information about 
nil- automorphisms of finite groups. This information will be obtained as a special case of a 
more general fact which, in turn, is a consequence of the following classical result of Baer [B] . 



Theorem 2.1 Let G be a group satisfying Max. Then the set of left Engel elements of G 
coincides with the Fitting radical of G. Consequently an Engel group with Max is a finitely 
generated nilpotent group. 
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Theorem A can now be proved easily. 

Theorem AlfG satisfies Max, and H is any group of nil- automorphisms of G, then [G, H] 
is contained in the Fitting radical of G and H stabilizes a finite subnormal series in G. In 
particular H is nilpotent. 

Proof Assume, first of all, that G is abelian. Let k be the rank of G and set T for the torsion 
subgroup of G. The subgroup T is finite and G/T is free abelian. For every prime p, the group 
A p = G/G p is elementary abelian and can be regarded as an F p - vector space of dimension at most 
k. Thus H p = H/Ch(A p ) is a unipotent group of automorhisms of A p , hence [A p ,kH p ] = 1. 
From this it follows that [G,/. H] < Dper G p = T. On the other hand T is finite and each 
element of H acts as a nil-automorphism, so that there exists m such that [T, m H] = 1. Thus 
[G,k+ m H] = 1 and H stabilizes a finite series in G. When G is nilpotent, H acts as a group of 
nil-automorphisms on each factor of the descending central series hence the above argument can 
be used to see that H stabilizes a finite central series of G. Let now G be any group satisfying 
Max, and choose h E H. It is clear that h is a left Engel element of the group G{h) which, in 
turn, satisfies Max. By theorem 12. li fe belongs to the Fitting radical of G{h). It follows that the 
subgroup [G, h] is contained in the Fitting radical of G. Therefore [G, H] is contained in the 
Fitting radical of G and property Max implies that [G, H] is finitely generated, hence nilpotent. 
Now, by the first part of the proof, H stabilizes a finite normal series in [G,H], showing that 
H stabilizes a finite subnormal series in G. By a well known result of Hall (see [Hl| ). H is 
nilpotent. □ 

As a corollary we obtain 

Corollary 2.1 Let G be a finite group and H a group of nil- automorphisms of G. Then H is 
nilpotent and [G,H] < Fit(G). 

We concentrate now on the main objects of our investigation, namely groups with residual 
properties. We start by recalling the definition of locally graded groups. 

Definition 2.2 A group G is said to be locally graded if every non trivial finitely generated 
subgroup of G has a proper normal subgroup of finite index. 

Of course any residually finite group is locally graded but the converse is not true. E.g. 
the groups PSL(2,F), where F is an infinite locally finite field, are locally finite (hence locally 
graded) but, being simple, they do not have proper subgroups of finite index. 

Question 1 has a positive answer for locally graded groups. 

Theorem B Let G be a locally graded group, and H a finite group of nil- automorphisms of 
G. Then H is nilpotent. 

Proof By way of contradiction we assume the claim false, so that there exist pairs (G, H) 
where G is locally graded, H < Aut(G) is finite and consists of nil-automorphisms, and H is 
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not nilpotent. Among them we select a pair (G, H ) in such a way that H has minimal order. 
Therefore every proper subgroup of H is nilpotent and the structure of H can be easily described. 
There exist two different primes p, q, such that H = Q(x) where Q is an elementary abelian 
g-group, x has order p m for some m, and Q is an irreducible (x)-module. 

In order to achive a contradiction, we shall prove that, for each i G N, if [g,i x] = 1 then 
[9->Q\ = 1- When i = this is true, because [g,ox] = 9- Assume we know the claim holds for 
all j < i, and choose g G G such that [g,i x] = 1. The element a = [g, x] satysfies [a,j_i x] = 1, 
therefore [a, Q] = 1. The subgroup A = [g -1 ,Q] is //-invariant. To see this we need only to 
show that A is normalized by x. In fact for any y G Q, we have 

\g-\yf = [[x,g]g-\y x ] = [a^g' 1 ^] = [g-\y x ] G ^ 

Assume A ^ 1. The group A is finitely generated, so that its finite residual R is a proper 
subgroup. Since unipotent groups of automorphisms of finite groups are nilpotent, the subgroup 
Q must act trivially on every finite image given by .//-invariant normal subgroups of A. Therefore 
Q acts trivially on A/R. Consider the map r\ : Q — > A/R defined by (y)' ? = [g~ 1 ,y]R. If u, v 
are in Q, then [<7~ 1 ,f,u] G R, hence 

{uvf = [g-\uv]R = [g-\vu]R = [g-\u}[g-\vfR = \g-\u\[g-\ v][g-\ v, u\R = («)»" 

The map r\ is therefore a surjective homomorphism, hence A/R is finite and R is trivial. Thus 
Q centralizes A. Moreover A is isomorphic to a quotient of Q and x acts on A as a unipotent 
automorphism. Since x has order p m the only possibility is that 1 = [A,x] = [g~ 1 ,Q,x]. Pick 
y G Q. Witt's identity gives 

l = \ 9 -\y-\x\y[y,x-\g- l ] x [x,g,yY>- 1 

On the other hand [5f _1 ,y _1 ,x] = 1 because [g~ l ,Q : x\ = 1, and [x,g,y] = 1 because Q 
centralizes a = [g,x]. Hence the above identity reduces to [y,x _1 ,(/ _1 ] = 1. This holds for every 
y G Q, thus [Q,x,g~ l ] = 1. But Q is an irreducible (x)-module, so that [Q,x] = Q, proving 
that Q centralizes g~ l , hence g. The inductive step is then complete. 

Choose g G G. There exists n = n{g) such that [g, n x] = 1. Thus [g, Q] = 1 and G; turns out 
to centralize G. Therefore Q must then be trivial and this contradiction proves that the claim 
holds. □ 

We focus now on Question 2. Since this question has a positive answer for finite groups, it 
seems natural to study the same problem for residually finite groups. 

To start off let us consider a finitely generated residually finite group G. It is easily seen that 
there exists a residual system Af, whose members are characteristic subgroups. The completion 
of G with respect to A/" gives rise to a profinite group G on which Aut(G) acts as a group of 
continuous automorphisms. For this reason it is convenient to consider Question 2 in the class 
of profinite groups and take advantage of the well-developed theory of profinite groups. 

For each N G Af indicate by iVo the closure of N in G. Each iVo is normalized by Aut(G) 
so that there is an induced action of Aut(G) on G/N = GN /N ~ G/N n G = G/N. If 
H < Aut(G) is a group of nil- automorphisms, then, for each iV G Af, its elements are nil- 
automorphims in their action on the finite group G/Nq. Therefore [G,H]Nq/Nq is nilpotent, 
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by corollary 12.11 It readily follows that the closure of [G, H] in G is pro-nilpotent and it is 
isomorphic to the cartesian product of its Sylow p-subgroups. This argument suggests that it 
might be useful, as a first step, to consider Question 2 for pro-p-groups. 

The first lemma we need is actually a fact about finite p-groups. 



Lemma 2.3 Let G be a solvable p-group of exponent p l and derived length d. If H < Aut(G) 
and [g, n h] = 1 for all g G G and h G H, then H has exponent bounded in terms of p l , n and d. 

Proof The proof is by induction on the derived length of G. 

When G is abelian we can invoke Lemma 16 of [U]. Assume now the claim holds for groups 
of derived length at most d— 1 > 1, and let G have derived length d. The group K = H^ p ' n > rf_1 ) 
acts trivially on both G/G^V and G^" 1 ). Thus [G,K,K] = 1 so that [g,x p '} = \g,xf = 1 for 
all g G G, x G K. Therefore, setting f(p l ,n,d) = f(p l ,n,d- l)p l , we get [g,h^ p ' n '^] = 1 for all 
g G G, h G H proving that exp(H) divides f(p l ,n,d), as claimed. □ 



Lemma 2.4 Let H be a group of n-unipotent automorphisms of the abelian group A. If [A,H] 
has finite exponent, then H has finite exponent. 

Proof It is enough to prove the lemma when the exponent of [A, H] is power of a prime p. If 
[A, H] has exponent p, define k by p < n < p . Then, given a G A and h G H, one has 

k k k 

1 = [a, n h] = a{h — l) n = a(h — l) p = a(h p — 1). Thus h p centralizes A and H has exponent 
dividing p k . Arguing by induction assume that the exponent of [A, H] is p r . Fix h G H and 
consider its action on the group A/[A,H] P . Since [A, H]/[A, H] p has exponent p r_1 , there 
exists m such that H m centralizes Aj [A, H] p . Thus [^4, H m ] < [A, H] p which has exponent 
p. Thus H m has exponent dividing p k , and H has exponent dividing mp k . □ 



The next lemma shows that, when a group of automorphisms of a p-group G is n-unipotent, 
then it is possible to construct several powerful subgroups of G. 



Lemma 2.5 Let G be a finite p-group, H < Aut(G) a group of n-unipotent automorphisms and 
L any H -invariant subgroup of G. Then there exists k such that R = [L, H k ] is powerful. 

Proof We treat the case p 7^ 2 first. Let k = f(p, n, 2) as defined in lemma [2T3l set K = H k and 
R = [L,K]. The group H acts on R as a group of n-unipotent automorphisms, hence [R, K] < 
R.WRP. Clearly [R, R9] < R( 2 )R p for every element g G L. Therefore R' = [R, [K, L}} < R^R P . 
Dedekind's modular law gives R' = R' D R^R P = R&(BP n R') but, since R^ is contained in 
the Frattini subgroup of R' , we get R' = RP n R' . Therefore R' < R p , proving the claim. When 
p = 2 choose k = /(4, n, 2) and apply the same argument. □ 

This lemma can be easily adapted to the case of pro-p-groups. Our main interest is the case 
of finitely generated pro-p-groups but it is better to consider a slightly more general setting, 
that we describe here below. 
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Definition 2.6 We say that (G,H) is a (n,p)-couple if 

1. G is a pro-p-group and H < Aut(G) is finitely generated; 

2. there exists n such that each h € H acts as an n-unipotent automorphism on G; 

3. G has a basis of open subgroups whose members are normalized by H . 

If G is a finitely generated pro-p-group and H is a finitely generated group of n-unipotent 
automorphisms of G, then (G,H) is a (n,p)-couple. In fact, for any fixed natural number k, 
G contains only finitely many subgroups of index k, so that, given N < G open, the normal 
subgroup Nh = C\heH ^ ls -H-invariant and of finite index. By a well known result of Serre, 
Nh is open (see [W2j . theorem 4.3.5) hence, if B is a basis of open subgroups for G, then 
Bh = {Nh I N £ B} is a basis of //-invariant open subgroups. On the other hand the set 
•Ah = {Ch(G/N) I N £ Bh} is a basis of open subgroups for H, with respect to which H is an 
Hausdorff topological group. 

When considering a topological group U, for every subgroup A, we shall indicate by clu(A) 
its topological closure in U. As usual a closed subgroup A is said to be finitely generated when 
A is the closure of a finitely generated discrete subgroup Aq . 

Lemma 2.7 Let (G,H) be a (n,p)-couple. Then the following facts hold 

1. There exists k such that, for every H -invariant subgroup L ofG, the group R = cIg([L, cli{{H k )]) 
is powerful. 

2. If d = d{H) is the minimal number of generators for H, there exists r = r(p,n,d) such 
that, for every x € G, the group L = clc(x H ) can be generated by r elements. 

Proof Let k = f(p,n, 2) or /(4, n, 2) whenp = 2, as defined in lemma |2"31 and set K = cl}{{H k )- 
Let B = {N\ | A € A} be a basis of normal /T-invariant open subgroups of G and choose L an H- 
invariant subgroup of G. For each AG A define L\ = LN\/N\ and R\ = [L, K]N\/N\ = [L\,K]. 
Since K = f] XeA H k C H {G/N x ), we have [L,K] < [L,H k }N x so that R x = [L,H k ]N x /N x = 
[L\,H k ]. Each L x is a finite p-group and, by lemma 1231 R x is powerful. Therefore the group 
R = cIg([L,K]), which is the inverse limit of the R x , is powerful. 

The group H/K is d-generated, residually finite and of exponent bounded by k, which 
is a function of p and n. By Zelmanov's solution of the restricted Burnside problem, H/K 
is finite of order bounded by a a suitable function r = r(p,n,d). Choose x € G and set 
L = clc(x H ). Arguing as in the first paragraph, we see that [L, K] < L^ 2 'L P and, in particular 
[L, K] < U LP < $>(L). Choose X = {hi \ i = 1, . . . , r} a left tranversal for K in H. Given any 
y € K and h € X, we have 

x h v = x h [ x h , y] = x h (mod $(L)) 

Thus, for each A € A, the set X x = {x h N x | h E X} generates L x = LN X /N X , modulo its 
Frattini subgroup. Thus (X x ) = L x . Since each L x can be generated by r elements, the same 
holds for their inverse limit L. □ 
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The next fact concerns the action of nil automorphisms on uniformly powerful group. 

Proposition 2.8 Let G be a finitely generated uniformly powerful pro-p-group and consider 
the usual additive structure defined on it. If a is a r-unipotent automorphism of G, then the 
automorphism induced by a on (G,+) is r-unipotent. 

Proof Given k > 1 and x £ G, define 

c(x, k) = x-VGR- 1 • • • x (-^ h - 1 L^h x (-^ h ^ jj x (-d'(»« 



We stick to the notation of chapter 4 of [DDMSj. Fix n > 1 and any x € G. The group 
G n+ i/G2n+2 is abelian, whence 

[x pn , k a] = C(x pn ,k) (mod G 2n +2) 

and the relation 

( x p n )(- i y(i) ak ' 1 = ( x (-ir(-K-')P n 

holds for every i = 0, 1, . . . , k. Thus 

(CV", k)f n = x« k + n x-(i)^- 1 +„ xO k - 2 +„...+„ x^ 1 ^ = a n (x, k) 

Lemma 4.10 of jDDMS] shows that, for each n, 

a n (x, k) = b n (x, k) = [x p ™ , k a] p " (mod G n ) 

so that the sequences {a n (x,k) \ n € uj} and {b n (x, k) \ n € a;} have the same limit. The limit 
of the first sequence is, by definition, the element 



4=1 i=\ ^J 



„a k ~ l 



x = [x, k a\ 



of (G, +). Since a is r-unipotent, [g, r a] = 1 for each g G G. Thus each b n (x,r) = 1, so that 
[x, r a] = in (G, +), showing that a acts unipotently on (G, +). □ 

At this stage we can prove some facts concerning unipotent automorphisms of nilpotent 
groups. These results will be used in the proof of the general case, but have some interest in 
their own. We begin by considering abelian groups. 



Proposition 2.9 Let (A,H) be a (n,p)-couple and assume that A is a torsion-free abelian 
group. There exists m = m(n) such that [A, m H] = 1. In particular H stabilizes a series of 
length at most m in A and it is nilpotent of class at most m — 1. 



Carlo Casolo and Orazio Puglisi 



Proof We refer to the terminology defined in |CT] . Let G = AH be the semidirect product 
of A and H . Thus A is a residually hypercentral n-Engel normal subgroup of G. Hence, by 
theorem 3 of |CTj . A^ n > < Cm.(n)(G) for suitable functions f,m. Being A torsion-free, it follows 
that A < Cm(n)(G), and H stabilizes a series of length at most m(n) in A. □ 

A straightforward consequence is the following. 



Corollary 2.2 Let (G,H) be a (n,p)-couple and assume that G is a finitely generated uniformly 
powerful pro-p-group. Then H is torsion-free and there exists m = m(n) such that H is nilpotent 
of class at most m. 

Proof Consider H as a subgroup of Aut((G, +)) = GL(r, Z p ). By proposition 12.91 H is a 
unipotent subgroup of GL(r, Z p ) so that it is nilpotent and torsion-free. Clearly ((G,+),H) is 
a (n,p)-couple so that, again bv 12.91 the nilpotency class H can be bounded in terms of n. □ 

The following is an easy fact that we shall need later. Its proof can be easily derived from 
the results about isolators collected in |H2|. 



Lemma 2.10 Let V be a torsion-free nilpotent group, U a normal subgroup of finite index and 
a an automorphism of V . 

1. The groups U and V have the same nilpotency class. 

2. If a centralizes U, then a = 1. 



Proposition 2.11 Let G be a finitely generated nilpotent pro-p-group, and H a d-generated 
group of n-unipotent automorphisms of G. Then the following hold 

1. H stabilizes a central series in G and is therefore nilpotent; 

2. there exists k = n(p,d,n) such that j K (H) is finite. 

Proof Under these hypotheses G has an //-invariant basis of open subgroups, so that (G, H ) is 
a (n,p)-couple. 

Assume first that G is torsion-free. It is easily seen, arguing by induction on the nilpotency 
class of G, that H is torsion-free. The key observation is that a nil automorphism of a torsion- 
free abelian group, has finite order if and only if it is the identity. Let K be the subgroup 
of H described in lemma I2TT1 and set R = cIg([G, K]). The group R is uniformly powerful 
because it is powerful and torsion- free (see theorem 4.8 in [DDMSJ). The group G acts on 
R by conjugation, and this action gives rise to an action on (R,+). Proposition 12.81 shows 
that each element of G acts unipotently on (R,+), whence G/Cg(R) is isomorphic to a closed 
group of unitriangular matrices over Z p . Each non-trivial closed subgroup of GL(r, Z p ) has rank 
bounded in terms of r whence, by theorem 3.13 of [DDMSJ, it possesses a characteristic powerful 
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subgroup of finite index. Such subgroup is uniformly powerful, because the unipotent subgroups 
of GL(r, Z p ) are torsion-free. Let M = M/Cq(R) be such a subgroup. The group H acts on both 
R and M/Cq(R) and we can use corollary 12.21 to see that there exists m = m(p,d,n) such that 
j m (H) acts trivially on both R and M/Cg(R). Set S = -y m (H) n K. Since S < Cgh(R), then 
[M,S] < [G,K] n C G (R) = R n Co(fl) = C(-R) • Notice that, by proposition E3 H stabilizes 
a finite series of length at most m in C(R)- Fix x £ M and consider the map 6 X : 5* — > C(R) 
defined by (s)9 x = [s,x]. It is readily seen that 9 X is an homomorphism. For any k £ K, s G S 
we have 

[s k ,x] = [s,x[x,/c~ 1 ]] fc = [s,x] k 

because [s, A; -1 ] belongs to R and S centralizes R. Thus # x is an homomorphisms of iC-modules, 
showing that S/ker(6 x ) has a if-central series of length at most m. The subgroup M has finite 
index in G, so that it is (topologically) finitely generated. Let x\, ■ ■ ■ ,x r be a set of generators 
for M. Then 
r 

p)ker(^) = {s £ S \ [s,Xi] = IVi = 1, . . . ,r} = {s e S \ [s,x] = 1VxGM} = C s (M) 
»=i 

By lemma [2 . 1UI C.<? (M) = Cs(H) = 1. For this reason 5 embeds, as a -ff-module, into the direct 
product ni=i 'S'/ker^J, so that S has a finite iT-central series of length at most m. This shows 
that j2m(K) = 1 and, by lemma 12.101 the same holds for H. Setting n(p,d,n) = 2m gives the 
claim. 

For the general case set T for the torsion subgroup of G and put G = G/T. If G is abelian, 
then T is pure in G, so that F = TnG p . Thus TG P /G P ~ T/T n(7 = T/T p and this shows 
that T is finitely generated, hence finite. By induction on the nilpotency class of G, we see that 
T is always finite and, in particular, closed in G. Thus G is a torsion-free pro-group and , if 
k = n(n,p,d) is the integer defined in the previous paragraph, the group J K (H) acts trivially on 
G. Any h € j K (H) is then completely determined, once we know the r elements th,i = [xi, h] so 
that |"y re (iT)| < \T\ r and the claim is established. □ 

This result has a rather strong consequence. 



Proposition 2.12 Let (G,H) be a (n,p)-couple with G a torsion-free nilpotent pro-p-group. If 
H can be generated by d elements, there exists k = K(p,d,n) such that J K (H) = 1. 

Proof Let k be the integer defined in proposition [27TTJ Given any i£G, the group L = cIq{x ) 
is, by lemma \277\ finitely generated. By proposition 12. 1 1 1 we have [L,7 K (if)] = 1, because G is 
torsion- free so that a non-trivial n-unipotent automorphism of G can not have finite order. In 
particular, j K (H) < Ch{x). Being x a generic element of G, this shows that J K (H) = 1. □ 

When the prime p is large enough, some useful bounds can be obtained. 



Proposition 2.13 Let G be a finite p- group of class m, and H a group of n-unipotent automor- 
phisms of G. There exists integers c = c(n,m), I = l(n) such that, if p > I, then H is nilpotent 
of class at most c. 
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Proof First of all assume G abelian. Set A = GH and let k = k(n), I = l(n) be the integers 
defined in lemma 3 of [CTj . For each g E G and a £ A, we have [g, n a] = 1 and [G, r H] = 1 for 
some r. Lemma 3 of [CT] can be used to get that, for every g € G and h\, /i2, • • • , ^fe £ -ff , one 
has [g,hi,h2,---,hk] = 1. Since p > / the element [5, /ii, /t2, • • • , hk] must be trivial, showing 
that [G,kH] = 1. Thus [G, 7fc(iT)] = 1 and the class of i7 is bounded by c(n, 1) = k(n) — 1. When 
G has class m> lwe consider the action of -ff on the factors of the ascending (or descending) 
central series and apply the above argument. It is readily seen that [G, m j. H] = 1 so that H has 
class at most c(n,m) = mk(n) — 1. □ 

This proposition can be immediately extended to pro-p-groups. 



Corollary 2.3 Let (G,H) be a (n,p)-couple, with G nilpotent of class m, and let l(n),c(n,m) 
be the integers defined in proposition \2.1'A If p > l{n), then H is nilpotent of class at most 
c(n,m) 

Proof If M is a basis of ^/-invariant open normal subgroups of G, then G/N is a finite p-group 
of class at most m, for all N 6 M. The group H acts on each G/N as a group of n-unipotent 
automorphism so that, by proposition 12.131 7c(n,m)+i (H ) centralizes all these quotients of G. 
Therefore [G, 7 c ( n ,m)+i (H)] ^ ClNeAf N = 1 ana - H nas c l ass a * most c(n,m). □ 



Before embarking in the proof of our main result, three technical lemmata are needed. The 
first of them is due to Hartley and its proof can be found in [HaJ . 



Lemma 2.14 Let G be a group with an ascending series with abelian factors. If the members of 
the series are characteristic, then G contains a characteristic subgroup U such that U is nilpotent 
of class at most two, and Cg(U) = C(U)- 



Lemma 2.15 Let G be a group and H a group of n-unipotent automorphisms of G. Suppose 
that H has a normal abelian subgroup A such that H = A(h), and assume that [G,A,A] = 1. 
Then H is nilpotent. 

Proof The subgroup M = [G, A] is a normal abelian subgroup of G, and will be regarded as 
an (h)-iaodule. For each k = 1, 2, . . . , n let X& = {g € G \ [g,k h] = 1} and set Aq = A, A^ = 
CA{Xk) when k = l,...,ra. The subgroups A^ form a descending chain in A and A n = 1. 
Choose g £ X± and consider the map 



%:A^ [G,A] 
a\ — > [a,g]. 
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An easy calculation shows that 9 g is an (/i)-module homomorphism, so that A/kei(O g ) is 
(fo)-isomorphic to a submodule of [G, A]. In particular h acts as an n-unipotent automorphism 
on A/ker(6 g ). Thus 

A = A / P| ker(9 g ) 
sex! 

embeds, as an (/i)-module, into a cartesian product of copies of M, hence it is acted upon by h 
as an n-unipotent automorphim. Therefore the element h stabilizes a series of length at most n 
in A) Mi- 

Suppose we have already shown that (h) stabilizes a series of length at most n, in each 
factor Ai/Ai + \ for % = 1, . . . , k < n. Let S be the semigroup generated by x = h , and ZS its 
semigroup algebra. Pick g € X^+i and define the map 

9 g :A k ^ M=[G,A] 
ai — > [a,g\. 

Using the fact that [g,h] £ Xj., we see that, for every a € A^, one has (a x )9 g = {(a)9 g ) x . 
Therefore Ak/hsi(9 g ) is isomorphic, as a ZS'-module, to a submodule of M. In particular, 
being (x — l) n € ZS, for every u £ A^/'kei{6 g ), the identity [u, n x] = 1 holds. Then the same 
identity holds for the action on A^/Ak+i, because this group embeds, as a ZS'-submodule, into 
a cartesian power of copies of M. Therefore (h) stabilizes a finite series of length at most n in 
Ak/Ak+i. Hence (h) stabilizes a series of length at most n 2 in A, showing that H is nilpotent. 
□ 



Lemma 2.16 Let (G,H) be a (n,p)-couple and assume that H = R(h) with R contained in 
the Hirsch-Plotkin radical of H . Let N < H be such that H/N is nilpotent and N(h) is locally 
nilpotent. If R is nilpotent then H is nilpotent. 

Proof The group H is finitely generated and metanilpotent hence, by 15.5.1 of [RJ, the Hirsch- 
Plotkin radical of H coincides with the Fitting subgroup and is nilpotent. The subgroup N(h) 
is subnormal in H because H/N is nilpotent. Thus N(h) is contained in R, the Hirsch-Plotkin 
radical of H. Hence (h) < R and from this it follows that H = R. □ 



Proposition 2.17 Let (G,H) be a (n,p)-couple, and let l{n),c{n,m) be the integers defined in 
proposition \2.13[ 

(1) Ifp> K n )> then [ G ^c(n,2)+i( H ),7c(n,2)+i{ H )} = 1 In particular 7c(n,2)+i(^) is abelian. 

(2) H is nilpotent. 

Proof Let N = {iVj | i G 1} be a base of i7-invariant open subgroups of G and set G{ = G/iVj. 
Each G L is a finite p-group so that, by lemma [2. 141 it has a characteristic subgroup Ui = Mi/Ni 
of nilpotency class at most two, such that G(j i (C/j) = C,{Ui). The group G = Yl ieI Gi can 
be topologized by assigning the set B = {Nj = Ylj eJ Gj | \I\ J\ finite } as a base of open 
neighborhoods for the identity. In this way G is a compact pro-p-group, and H acts on it as a 



12 Carlo Casolo and Orazio Puglisi 

group of n-unipotent automorphisms. The subgroup U = Yl i£l U% is closed in G, is acted upon 
continuously by G and H, and Cq(U) = Q{U). Choose any cofinite subset J of I and define 
Uj = UCiNj = Y\j € j Uj. The subgroups Uj form a basis of open neighborhoods of the identity 
in U. 

If A = C H (Uj), we have that [G,A] < C G (Uj). Thus [Uj, [G,A]Nj/Nj] = 1 for all j E J, 
hence [G^jNj/Nj < C Gj {Uj) = ((Uj) when j E J. Hence 

[\G,A]N J /N V A] < [C(U 3 ),A] < [U 3 ,A] = 1 

for all j E J, showing that [G, A, A] < Dj^jNj = 1 which, in turn, proves that A is abelian. 
In particular, if C = Ch(U), then [G,C,C] = 1 and C is abelian. If p > l(n) we can apply 
corollary 12.31 to the action of H/Cjj(U) on U, showing that r y c (n,2)+i(H) < Cjj(U) so that (1) 
holds. 

Let X = {h E H \ h is abelian }. Assume that U = (J/ie-fAX Cu(h)- Since H consists of 
continuous automorphisms of U, each Cjj(h) is closed and, being H \ X finite or countable, we 
may invoke Baire's theorem to prove that, for some h € H\X, the subgroup Cjj(h) contains an 
open subset. Therefore Cu{h) must contain an open subgroup of the form Uj. The subgroup Uj 
is normalized by H, so that U j < Cjj{h x ) for all x € H showing that U j centralizes h . Thus 
h < Cjj(Uj) and, by the previous paragraph, h is abelian, a contradiction. It is therefore 
possible to find u € U\[J h&H \ X Cu{h), so that Cjj(u) < X. The group V = cljj{u H ) is nilpotent 
of class at most 2, it is finitely generated ( see lemma l27T|) and Ch(V) is contained in Ch(u). 
Since CuiV) is contained in X, for every element c E Ch(V) we have that c H is abelian and 
it follows that Ch(V) is a 2-Engel group. In particular CuiV) is nilpotent of class at most 3. 
Moreover proposition 12.111 shows that H/Ch{V) is nilpotent. Thus H is metanilpotent and, in 
particular, it is solvable. 

To complete the proof we argue by contradiction, selecting H a counterexample of minimal 
derived length. Under this assumption each finitely generated subgroup of H' is nilpotent, so 
that H 1 is contained in R, the Hirsch-Plotkin radical of H. However H is metanilpotent hence, by 
15.5.1 of [RJ, R is nilpotent and coincides with the Fitting subgroup of H. Each counterexample 
of minimal derived length, is therefore nilpotent-by-abelian. 

We may choose H in such a way that d(H/R), the minimal number of generators of H/R, 
is minimal. If H/R = (h\R, . . . , h[R) and I > 2, then the two subgroups H% = (h\,R) and 
H2 = {h,2i ■ ■ ■ hi,R), are both normal and nilpotent. To see this let F be a finite subset of Hi 
and consider W = (F). Clearly Fit(VF) > W D R, hence W/Fit(W) is an homomorphic image 
of WR/R < Hi/ R, showing that H / /Fit(H / ) has minimal number of generators strictly smaller 
than d(H/R). By assumption W is nilpotent. The group Hi is then locally nilpotent and normal 
in H , thus it is contained in R, the Hirsch-Plotkin radical of H , proving that H = H1H2 < R, a 
contradiction. Therefore d(H/R) = 1 and it is possible to write H = R(h) for a suitable h E H. 
Moreover we may choose H in such a way that the nilpotency class of R is minimal. Consider 
S = Ch(cIu{u H )) and the subgroup S(h). Since H/S is nilpotent, then S(h) is subnormal in H 
and, by lemma [2.161 it can not be locally nilpotent. If, for every s E S, the subgroup (s, h) is 
nilpotent, it is easy to show that S{h) is hypercentral, hence locally nilpotent. For this reason 
there must exist a E S such that (a, h) is not nilpotent. Since a E S C X, then a' a,fe ' is abelian. 
For this reason there is no loss of generality in assuming that H = (a, h) = A(h) where A = a . 
Notice that A < R. 
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The group H is then metabelian, so that it satisfies Max-n, the maximal condition on normal 
subgroups. 

If W is a pro-p-group, and a : H — > Aut(W) is an homomorphism, we say that a is an 
n-unipotent representation if (W, H a ) is a (n,p)-couple. 

The set 

N = {N \ N < H is the kernel of an n-unipotent representation and H/N is not nilpotent} 

is not empty, so that it has a maximal element M. By looking at the representation related to 
M, we may assume that 

• if 1 7^ N < H and H/N acts faithfully on a pro-p-group W in such a way that (W, H/N) 
is a (n, p)-couple, then H/N is nilpotent. 

Denote by C the centralizer of U in H. 

Two cases should be considered. 

Case 1. C is not trivial. 

Thus H/C is nilpotent and, as we have seen before, C is abelian and contained in R. Since 
[G,C,C] = 1, we may apply lemma [2.151 to see that the subgroup C{h) is nilpotent. Lemma 
12.161 can now be used to see that H is nilpotent, a contradiction. 

Case 2 C = 1. 

The group -ff acts faithfully on U. If f7 is torsion, then it must be of finite exponent, because 
it is nilpotent and compact. If this is the case, for each u € U, the subgroup cl(u H ) has, by 
lemma [27TI a uniformly bounded number of generators. By Zelmanov's solution of the restricted 
Burnside problem, cl(u H ) has uniformly bounded order. In particular cl(u H ) = u H and, for 
some m, H m < Ch{u H )- Hence 

H m < f| C H {u H ) = C H (U) = 1 
ueu 

and H is finite, hence nilpotent. Thus U is not a torsion group. 

When W is a nilpotent group, we shall indicate by T(W) its torsion sugbroup. Let S = 
Ch(U/T(U)) and, for each s£S, consider the homomorphism 



^ s : U — > C//C(t/) 
n^ [u,s]C(l7). 

This homomorphism is continuous and for this reason \U , s\(,(JJ) / (,{JJ) is compact. On the 
other hand [U, s] is contained in the torsion subgroup of U, hence it has finite exponent, say e\. 
For this reason [cl(U ei ), s] < C(U)- The same argument applies to the continuous homomorphism 



<p s : du(U ei ) — ► ((U) 
U I — > [u, s] 
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proving that clu{U ei ) e2 < ker(ip s ). It is then straightforward to see that, for each s G S, there 
exists e = e(s) such that [clu(U e ),s] = 1. The subgroup S can be generated, as a normal 
subgroup, by finitely many elements si,...,s/. Choosing e = max{e(sj) | i = 1,...,/}, we 
get [du{U e ),S] = 1. If S + 1 then P = C H {du(U e )) > S is not trivial and therefore #/P 
is nilpotent, by our choice of H. Thus j a (H) < P for some a. The group H acts obW = 
U/clu(U e ), (W,H/Ch(W)) is a (n,p)-couple and W is nilpotent of finite exponent. As we have 
seen before, H/Ch(W) is a finite p-group. There exists b such that jb{H) < Ch{U /cljj{U e )) and, 
if c = max{a,6}, then P = j c (H) < C H {U/du(U e )) nC H (du{U e )). In particular [U,D,D] = 1 
and D is abelian. By lemma |2. 151 the subgroup P(/i) is nilpotent and, by lemma [2.161 H would 
be nilpotent. For this reason the subgroup S must be trivial. 

For every u € U set L(u) for the closure in U of u , and C(u) = Ch(L(u)). Recall that each 
L(u) can be generated by a uniformly bounded number of elements. If C{u) = 1 for some u, 
then H is nilpotent, by proposition 12.111 a contradiction. Thus C(u) is always non-trivial, and 
H/C(u) is nilpotent, because L{u) is a finitely generated pro-p-group, hence countably based. 
By proposition 12. Ill there exists k such that ^y K (H)C(u)/C(u) is finite. Therefore [L{v),^ K {H)\ 
is contained in T(L{u)) < T(U). It then follows J K (H) < S = C H (U/T(U)) = 1, so that H is 
nilpotent. This final contradiction proves that the claim holds so that H must be nilpotent. □ 

We are now in a positon to prove Theorem C. 

Theorem C Let G be a profinite group and H a finitely generated group of n-unipotent auto- 
morphisms. Assume that G has a basis of open normal subgroups whose members are normalized 
by H. Then F = clc([G,H]) is pro-nilpotent and H is nilpotent. 

Proof The fact that F is pro-nilpotent has been already noticed in the paragraph before lemma 
12.31 Write F = Y\ peI F p where each F p is the p-Sylow subgroup of F. Assume first that H 
acts faithfully on F. By theorem 12.171 each H/Ch(F p ) is nilpotent, say of class c p and, when 
P > K n )> [-Pp,7c(n,2)+i(-H') ) 7c(n,2)+i(-ff)] = 1- Therefore, if c = max{c(n,2), c p \ p < l(n)} + 1}, 
we get [P, j c (H) , j c (H)] = 1 showing that H is abelian-by-nilpotent. 

If, by way of contradiction, we assume the claim false, the same argument used in the proof of 
theorem 12.171 can be applied, to see that there must exist a counterexample H that is nilpotent- 
by cyclic. Write H = R(h) with R the Fitting subgroup of H. Lemma 12. 151 savs that j c (H)(h) 
is nilpotent, hence H is nilpotent, by lemma [2.161 a contradiction. 

Therefore H can not act faithfully on P and C = Ch(F) is not trivial. We point out 
that the previous argument shows that H/C is nilpotent. The group C satysfies [G,C,C] = 1, 
whence H is abelian-by-nilpotent. Assuming the claim false we apply once again the argument 
of theorem I2.17[ to show that H may be assumed to be nilpotent-by-cyclic. At this stage the 
same considerations of the previous paragraph provide a contradiction, thus proving that H 
must be nilpotent. □ 

As anticipated in the introduction, our results imply a well known theorem due to Wilson 

Theorem 2.18 Let H be a finitely generated residually finite n-Engel group. Then H is nilpo- 
tent 
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Proof Let G be the profinite completion of H. Since H/Q(H) is a finitely generated group of 
n-unipotent automorphisms of G, we apply theorem C to show that H/Q{H) is nilpotent. Thus 
H is nilpotent and the theorem is proved. □ 
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